1. Technical Field
The invention is related to a system and process for generating a panoramic video of a scene, and more particularly to such a system and process that employs a multi-camera rig to capture individual videos that collectively depict the surrounding scene, and which are then stitched together on a frame by frame basis to form the frames of the panoramic video.
2. Background Art
A panoramic video is a video made up of a sequence of panoramic frames depicting a surrounding scene. Ideally, the panoramic video makes available a seamless, 360 degree, view of this scene. In this way, a person viewing the panoramic video can select different portions of the scene to view on a real-time basis. In other words, a person viewing the panoramic video on the proper viewer can electronically steer his or her way around in the scene as the video is playing.
A number of different systems for generating panoramic videos have been previously developed. For the most part, these systems employ a mirror arrangement to capture the surrounding scene. For example, one existing system, referred to as a catadioptric omnidirectional camera system, incorporates mirrors to enhance the field of view of a single camera. Essentially, this system, which is described in a technical report entitled “Catadioptric Omnidirectional Camera” (Shree K. Nayar, Proc. of IEEE Conference on Computer Vision and Pattern Recognition, Puerto Rico, June 1997), uses a camera that images a hemispherical mirror to generate a panoramic still image with a 360°×210° field of view. Another similar mirror-based system unwarps a spherically distorted video produced by the mirror-and-camera rig into a rectangular video stream then encodes it using standard streaming authoring tools. The person viewing a video produced via this system sees a sub-region of the scene captured in the panoramic video and can pan within the scene. While these mirror-based single camera systems are capable of producing convincing panoramic stills and video, they suffer from a relatively low resolution and a fairly complex camera rig owing to the mirror arrangements.
Another current panoramic video system that attempts to overcome the resolution and complexity problems, foregoes the use of a mirror, and employs a multiple camera head instead. The head consists of six cameras mounted on the six faces of a 2-inch cube, resulting in a 360°×360° field of view. The system also provides post-processing software to stitch the video streams from the individual cameras into a panorama. This multi-camera system has higher resolution than the catadioptric systems described above, but has the disadvantage of an expensive stitching stage and parallax artifacts due to the cameras not sharing a common center of projection.
One other system of note employs both a mirror arrangement and multiple cameras in an attempt to achieve a higher resolution without the stitching and parallax problems of the non-catadioptric, multi-camera system just described. Essentially, this system uses the mirror arrangement to create a common effective viewpoint for the cameras. While this system improves the resolution and reduces the aforementioned stitching and parallax problems, it still requires the use of a complex mirror-and-camera rig.
The present invention is directed at a non-catadioptric, multi-camera system and process that is capable of providing high resolution panoramic video, minimal stitching and parallax problems and a relatively simple camera rig.